Papers with few-shot learners
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work shows that large-scale pretrained language models (PLMs) are effective few-shot learners. |
| Approach: | They propose a method that treats few-shotlearners as crowdsourcing workers . they propose to use these workers to train models that solve a task well . |
| Outcome: | The proposed approach treats few-shotlearners as crowdsourcing workers . the resulting annotations can be utilized to train models that solve the task well . |
DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Meta-learning has not yet succeeded in NLP due to the lack of a well-defined task distribution . meta-learners tend to overfit their adaptation mechanism and datasets are heterogeneous . |
| Approach: | They propose a method for decomposing datasets into Reasoning Categories to form additional high quality tasks. |
| Outcome: | The proposed method improves the accuracy of meta-learners by 1.5-4% across four few-shot NLI problems. |
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting (2023.eacl-main)
Copied to clipboard
| Challenge: | Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. |
| Approach: | They propose to use frozen unimodal models to learn a lightweight mapping between the representation spaces of unimod models using aligned image-text data. |
| Outcome: | The proposed method can generalize to unseen VL tasks from a few in-context examples while training orders of magnitude fewer parameters. |
Pre-trained Token-replaced Detection Model as Few-shot Learner (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained masked language models have demonstrated remarkable few-shot learning ability . a novel approach to few- shot learning with pre-tried token-replaced detection models is proposed . |
| Approach: | They propose a method to reformulate a classification or regression task as a token-replaced detection problem by using pre-trained token-based models. |
| Outcome: | The proposed approach outperforms pre-trained masked language models in learning tasks . it can learn models with a few examples and generalize well from limited examples like humans . |
Reordering Examples Helps during Priming-based Few-Shot Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning from limited data are not efficient . we show that presenting examples in the right order is key for generalization . |
| Approach: | They propose a method to learn from limited data using examples as prompts . they propose PERO, which uses examples as search over set of permutations . |
| Outcome: | The proposed method can generalize using as few as 10 examples, the authors show . it can be used on sentiment classification, natural language inference and fact retrieval tasks . |